Section: New Results

Motion Sensing and analysis

Sensing human activity fo detecting falling motions

Participants : Franck Multon [contact] , Richard Kulpa, Anthony Sorel, Edouard Auvinet.

Sensing human activity is a very active field of research, with a wide range of applications ranging from entertainment and serious games to personal ambient living assistance. MimeTIC aims at proposing original methods to process raw motion capture data in order to compute relevant information according to the application.

In personal ambient living monitoring, we have collaborated with University of Montreal, Department of Computer Science and Operations Research (DIRO) which main activity is biomedical engineering. A co-supervised student is addressing two complementary problems: detecting people falling in everyday environment and providing easy-to-use clinical gait analysis systems for early detection of potential risks of falling. These two problems have been addressed by reconstructing the visual hull of a subject according to synchronized classical of depth cameras. As visual hull is based on videos it's subject to occlusions which generally occur in natural environment, such as a room with furniture. We have adapted the classical visual hull algorithm in order to be less sensible to occlusions. We also have proposed an index based on 3D silhouette vertical distribution which enhance this property to tackle occlusion problems [1] . This index is based on a ratio: the volume above a given threshold divided by the total body volume. It has been successfully applied to dozens of falling scenarios involving natural occlusions with furniture. The second problem consists in extracting relevant indexes in gait that could enable clinicians to identify elderly people who have a risk of falling. Classical indexes are based on gait regularity and asymmetry in dual tasks protocols (such as walking while counting downward). 3D silhouettes intrinsically contain all the required information in a unique representation contrary to multi-point motion capture systems. However extracting the relevant information from 3D volumes is complex. We have proposed an original approach based on statistical analysis of the volumes in order to compute indexes for gait asymmetry while simply using 3 depth-cameras (Microsoft Kinects) [1] (see figure 4 ).

Figure 4. 3D silouhettes reconstructed with three depth-cameras in order to analyze gait asymmetry.

In entertainment and serious games, the problem is different as we need to accurately now the action performed by the user in order to react in a convenient manner. Collaboration with Artefacto Company enabled us to develop such motion recognition methods in serious games scenarios. Given motion capture data provided by an optical motion capture system lead to large state vectors in which the relevant information is hidden. Mixture of Gaussians is generally used as an input of Hidden Markov Models to recognize a motion according to this raw data. To simplify, features are generally introduced in order to capture the relevant geometrical property of the motion with either general information (such as joint angles or Cartesian positions) or application-specific information. The former type of information has the advantage to be generic but leads to recognizers that are very sensitive to style and morphology variations. We have proposed a new generic feature based on morphology-independent representation that enables to tackle this problem (submitted to Eurographics2012). The recognition rate is above 75% for very similar upper-limb motions (see figure 5 ) while classical methods fail to recognize the same motions (recognition rate below 50%).

Figure 5. The motion of a user equiped with reflective markers is automatically recognized with a HMM system based on morphology-independent features.

The Joyman: a novel immersive locomotion device for virtual environments

Participant : Julien Pettré [contact] .

We proposed a novel interface called Joyman, designed for immersive locomotion in virtual environments [18] . Whereas many previous interfaces preserve or stimulate the users proprioception, the Joyman aims at preserving equilibrioception in order to improve the feeling of immersion during virtual locomotion tasks. The proposed interface is based on the metaphor of a human-scale joystick. The device has a simple mechanical design that allows a user to indicate his virtual navigation intentions by leaning accordingly. We also propose a control law inspired by the biomechanics of the human locomotion to transform the measured leaning angle into a walking direction and speed - i.e., a virtual velocity vector. We aim at using this interface to enable natural interaction with virtual humans with low-cost devices. The Joyman is patented and was presented at the Emerging Technologies, Siggraph Asia, Hong-Kong [20] .

These are joint results with the VR4i team (Anatole Lécuyer and Maud Marchal).